BubbleStorm: replication, updates, and consistency in rendezvous information systems

As distributed systems are getting more and more complex, search facilities for finding services and data within the system become crucial. Users expect search engines to deal with complex query languages like keyword search, SQL, or XPath. At the same time, application developers cannot be expected to come up with distributed versions of those query languages from scratch. Rendezvous search systems are a very scalable solution to this problem. By separating the query processing from the network communication, existing libraries for query processing can be easily reused. A wide range of rendezvous search systems for different scenarios has been proposed in the past. Their scalability and resilience make them an excellent choice for search in large-scale and dynamic peer-to-peer environments. The resilience stems mainly from the high number of replicas per datum, which however makes replica maintenance difficult. Unfortunately, most rendezvous search systems lack maintenance algorithms to sustain the desired replica count under node churn. Replica maintenance is closely related to update mechanisms for mutable data. The highly distributed nature of peer-to-peer systems in general and the high replica count of rendezvous search systems in particular require carefully designed mechanisms for consistent updates with concurrent accesses. In this thesis, replica maintenance and update mechanisms for the BubbleStorm peerto-peer overlay and related rendezvous search systems are introduced. After analyzing the design space of replica maintenance for peer-to-peer systems, a complete solution covering all identified use cases is presented. This includes a maintainer-based mechanism for data managed by a single node and a collective mechanism for data that shall be persistent beyond any particular node’s session time. The algorithms are evaluated in BubbleStorm’s sophisticated testbed, which allows prototype experiments and simulations with the same source code.